Depth cameras may obtain depth information for an environment, which may be used to generate a 3D model of the surfaces and objects within the environment. As surfaces and objects change and move over time, new depth information may be used to update the 3D model.